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for their present roles. Findings included: nearly half the 
respondents indicated that their writing ability was a factor in 
obtaining their present job, and on average they spent nearly a 
fourth of their work week in writing tasks; among the respondents who 
undertook graduate studies, over a fourth had authored or co-authored 
a publication and a fourth had written grant proposals; and 
respondents reported they also spent time each week writing for 
pleasure. Questions about the college's writing program concerned: 
tho types of writing experiences that helped the students prepare for 
writing after graduation; the frequency of specific guidelines being 
provided for writing assignments; the extent that revision was 
emphasized; the extent feedback on writing was provided; how many 
papers over and under 10 pages were written; and the types of 
teaching methods which were used in class. (Contains 11 references.) 
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ASSESSING A WRITING-ACROSS-THE-CURRICULUM PROGRAM 
WITH A SURVEY OF RECENT GRADUATES 

ABSTRACT 

When a writing-across-the-curriculum program has been in existence for over 
a decade, it is possible to gather information regarding the program's effectiveness 
from graduates in the workplace and in graduate/professional schools. This paper 
describes one such effort, using a survey of 2,000 graduates from eight graduating 
classes to gather data regarding the types of writing being utilized by the 
graduates, and the effectiveness of their writing curriculum in preparing them for 
their present roles. Institutional researchers will find this paper useful where such 
programs are already in place or are being contemplated. 
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Ar<^ESSIIMG A WRITING-ACROSS-THE-CURRICULUM PROGRAM 
WITH A SURVEY OF RECENT GRADUATES 

Introduction 

Despite the growth in tl^e number of colleges and universities establishing 
writing-across-the-curriculum programs, and the development of professional 
literature supporting these initiatives, there has not been much sharing of a 
methodology to evaluate the long-term effect of these writing programs. With their 
growing popularity, there is also an increasing need to assess these programs to 
determine the manner in which students have benefited from the program and to 
ascertain areas in which the programs need to be strengthened. 

The literature of program review and assessment (eg,, Ewell, 1988; Kinnick, 
1985; Wilson, 1987) emphasizes the importance of evaluation of institutional 
effectiveness at the program level. Specifically, McLeod and Soven (1991) urge 
directors of writing programs to build evaluation into their program. Multiple 
measures, including student surveys, are recommended. Others who provide 
suggestions for such evaluations include Davis, Scriven, and Thomas (1987), 
Fulwiler (1984 & 1988), White (1985 & 1989), Witte and Faigley (1983), and 
Young e:id Fulwiler (1986). 

This paper shares the methodology and general results of a survey which 
assessed the effects of a writing-across-the-curriculum program on graduates two 
to nine years after program completion at Saint Mary's College (Indiana). A 
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Catholic women's college, Saint Mary's is a competitive, highly-ranked, 
comprehensive college in the liberal arts tradition. The College was among the first 
to incorporate the writing-across-the-curriculum program, beginning in the early 
1970s, and its program was cited in the 1988 US News (26 Oct 1987) for the way 
the college placed emphasis on developing its students' communication skills. 
Program faculty have been invited to describe this program on many other 
campuses throughout the years. 

Initially, the Writing Program was directed from within the English 
department; however, in the past decade it has more widely involved faculty 
across the disciplines. Since the late 1980s, students have been required to meet 
two writing requirements as part of their graduation requirements. As freshmen, 
they take at least one course from the general education core designated as a 
writing class, developing a writing portfolio of at least three required pieces 
reviewed by a cross-disciplinary reading committee. Approval of this portfolio 
earns them their "Basic W". An "Advanced W" is earned during the senior year, 
with criteria varying among the majors. Again, this frequently is a portfolio review 
of one to four pieces, evaluated by faculty members within the respective 
department. The initiation of the "Advanced W" requirement was the last 
modification to this program's evolution. 

Presently, oversight of the Writing Program rotates among a number of 
faculty, with different co-directors selected each semester. Often one of these co- 
directors is someone who has co-directed the program in the past, providing 
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continuity among the leadership. A monthly workshop open to all college faculty 
provides opportunity to present and discuss a wide range of topics relating to 
writing. 

A wealth of anecdotal information regarding the usefulness and success of 
the Writing Program has filtered back to the faculty since the program had been in 
effect, but no formal evaluation had ever been made of the program. A faculty 
committee consisting of several veteran writing faculty, including some with 
psychometric and survey design experience, was assigned the task of working with 
the Saint Mary's institutional researcher to develop a research methodology for 
assessing the program. It was quickly determined that a true experimental design 
comparing the post-baccalaureate experience of the Writing Program graduates at 
Saint Mary's with matched peers from one or more other colleges where there was 
no writing program would be virtually impossible to administer. Hence, the 
resulting plan was to gather descriptive information in a survey of eight 
consecutive and representative graduating classes overlapping the last major 
revision in the program's requirements. 

A one-page survey (front and back, see Appendix A) was developed by this 
group of writing instructors and the institutional researcher. The instrument was 
divided into two major sections: the first part focused on the assessment of 
writing since graduation (in the workplace, in graduate/professional school, and for 
pleasure); the second assessed their undergraduate writing experience -the nature 
of their writing instruction and the extent to which writing pervaded the curriculum. 
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In gathering information regarding the writing activities in which respondents 
had engaged since graduation, somewhat parallel information was sought. What 
types of writing were they doing? Was their writing ability recognized by those 
with whom they associated? How much time did they spend in writing-related 
activities? And which of the writing experiences at the college had helped prepare 
them for their writing activities? 

As the committee drafted this survey, it continuea to solicit feedback 
regarding the appropriateness of the items and the inclusiveness of as many 
evaluative aspects of the program from the writing faculty. This also gave 
widespread ownership of the survey and researcli design to the writing faculty. 

Methodology 

The survey was mailed to a randomly-selected sample of 2,000 graduates 
from the classes of 1985 through 1992 (250 in each of the eight classes), along 
with a cover letter from the chief academic officer and a postpaid business reply 
envelope. A label indicating the subject's graduating year, degree, major(s), and 
address was affixed to each form. The classes were selected because the Writing 
Program had undergone its last change in the mid-point of this time period wihie 
these students had been enrolled at Saint Mary's. 

Responses were received from over a third of the sample, and appear to be a 
good representation of the population (see Table 1). There was a slight over- 
representation from the two most recent classes; ironically, these classes have the 
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Table 1 



Response rate of Writing Program Survey- by class 





Total 


No. 






% of 


% of 


% of 




i n 


in 


% of 


No. 


entire 


class 


total 




class 


survey 


s<;mple 


rasp. 


class 


sample 


rasp. 


Class of 1985 


391 


200 


12.5% 


73 


18.7% 


36.5% 


11.1% 


Class of 1986 


383 


200 


12.5% 


76 


19.8% 


38.0% 


1 1 .6% 


Class of 1987 


370 


200 


12.5% 


80 


21.6% 


40.0% 


12.2% 


Class of 1988 


385 


200 


1 2.5% 


76 


1 9.7% 


o o r\o/ 

J8.0% 


11 CO/ 

1 1 .D70 


Class of 1989 


391 


200 


12.5% 


82 


21.0% 


41.0% 


12.5% 


Class of 1990 


392 


200 


12.5% 


77 


19.6% 


38.5% 


11.7% 


Class of 1991 


436 


200 


12.5% 


93 


21.3% 


46.5% 


14.2% 


Class of 1992 


41 1 


200 


12.5% 


99 


24.1% 


49.5% 


15.1% 


Total 


3,159 


1,600 


100.0% 


656 


20.8% 


41.0% 


1 00.0% 



shortest post-baccalaureate experience and would have the 'east amount of activity 
to report and the least experience to evaluate. Table 2 shows the response rate by 
major, showing more variation among the subgroups. Over a fourth of the 
education majors responded to the survey, while fewer than a sixth of the 
humanistic studies, psychology, philosophy, and communication/theatre majors 
responded. 

The quantifiable responses were analyzed using SPSS; writing faculty 
developed a coding scheme to categorize subjective responses, and these were 



Assessment of writing program 

8 



Table 2 

Response rate of Writing Program Survey — by major 







Mijmhsr 






Tntal 


of 


Resp 




firaduatp^ 

V-i 1 Ci vl VI C4 L O 


Respondents 


Rate 


Art 


1 1 1 


21 


18.9% 




1 55 


34 


21 .9% 


Business 


755 


160 


21.2% 


r^h Amic1"r\/ 
v>i loi 1 iio 11 y 


58 


1 3 


22.4% 






64 


16.6% 


Economics 


93 


21 


22.6% 


Education 


282 


79 


28.0% 


English 


402 


80 


19.9% 


History 


207 


38 


18.4% 


Humanistic Studies 


167 


26 


15.6% 


Mathematics 


127 


28 


22.0% 


Modern Language 


115 


23 


20.0% 


Nursing 


230 


46 


20.0% 


Philosophy 


61 


10 


16.4% 


Political Science 


261 


61 


23.4% 


Psychology 


245 


39 


15.9% 


Sociology/Social Work 


150 


27 


18.0% 


Other 


41 


8 


19.5% 



also included in the SPSS analysis. In a separate data file, the description of each 
respondent's "most instructional writing experience" was entered along with 
demographic identifiers for subgroup analyses. 

Results 

Some of the results from the Writing in the Workplace section are shown in 
Table 3. Nearly half (47.9 percent) of the respondents indicated that their writing 
ability was considered as a factor in their obtaining their present position, and on 
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Table 3 

Writing in the Workplace, Graduate/Professional Schools, and for Pleasure 



Writing in the Workplace 

Was your writing ability considered as a factor in your obtaining your present position? 

Number Percent 
Yes - 291 47.9% 

No - 316 52.1% 

Not working - 49 

On average, about how many hours per week do you spend in work-related writing tasks? 
Mean 1 1 .47 hrs/week (Std Dev 10.52) 

Median 8.00 hrs/week 

Range 0 to 60 hrs/week 



Writing in Graduate/Professional Schools 



How many publications have you authored or co-authored? 



Mean 1 .49 (Std Dev 8.55) 

Median less than 1 

Range 0 to 10 

How many grant proposals have you written or helped write? 

Mean .97 (Std Dev 4.42) 

Median Less than 1 

Range 0 to 50 

About how many hours per week do you spend writing? 

Mean 8.8 hrs/week (Std Dev 1 1.07) 

Median 5 hrs/week 

Range 0 to 93 



Writing for Pleasure 



About how many hours per week do you write for pleasure? 
Mean 2.43 hrs/week (Std Dev 2.31) 

Median 2 hrs/week 

Range 0 to 30 

Do friends and family solicit your help In their writing tasks because they think you are a good 
writer? 

Number Percent 
Yes - 436 70.8% 

No 180 29.2% 

No response - 40 
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average these respondents spend nearly a fourth of their work week (median of 8 
hours, mean of 1 1 .47 hours) in writing tasks. 

There were 286 respondents who answered questions related to graduate/ 
professional schools, and their results are also shown in Table 3. There appears to 
be a fair amount of scholarly writing activity by these students. Over a fourth had 
authored or co-authored a publication, and a sixth had done so two or more times. 
Similar evidence is found for writing grant proposals, with a fourth having done this 
at least once. Most of these respondents spend around five to eight hours writing 
each week. 

Respondents also spend time each week writing for pleasure (see also in 
Table 3), although this time is often limited to two or three hours weekly. To a 
widfc extent (in more than 70 percent of the cases), friends and family solicit 
respondents' help in their writing tasks because they are perceived as being good 
writers. Whether these results are typical or not, there is evidence that Saint 
Mary's graduates are still utilizing the writing skills they honed while in college. 

The evaluation of the Writing Program also included indicating on parallel 
series of writing experiences those which may have helped the students prepare for 
the variety of writing they would do after graduation. These are shown in Table 4. 
The experience cited most often for preparing respondents for writing in the 
workplace and writing in school were the papers written for classes in their major, 
followed by the "Basic W", the senior comprehensive (in many cases this consists 
of a major research paper), the papers written for classes in their minor, and the 
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Table 4 



Which writing experiences at Saint Mary's helped prepare you? 





Writing 


Writing 


Writing 




in the 


in 


for 




Workplace 


School 


Pleasure 


Basic W 


405 


184 


329 


Papers written in 








courses in your major 


478 


238 


263 


Papers written in 








courses in your minor 


228 


110 


147 


Advanced W 


162 


73 


89 


Senior comprehensive 


235 


131 


117 


Other 


108 


36 


91 


None 


32 


3 


112 



"Advanced W'\ The lower rating of the latter two experiences is likely due to the 



fact that many students at Saint Mary's do not graduate with minors, and the 
"Advanced W" was not required of at least half of the survey population. A 
different ranking of these experiences in terms of their preparation for personal 
pleasure writing swapped the importance of the top two reasons cited for the 
workplace and school, with the "Basic W" being cited most often as a preparation 
for personal pleasure writing. The senior comprehensive also dropped in ranking, 
and there was a higher number of respondents indicating that none of the writing 
experiences at Saint Mary's helped prepare them for this writing, or they cited 
other experiences. 

The survey also gathered further information regarding these writing 
experiences, and these results are shown in Table 5. Over two-thirds (68.6 

u 
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Table 5 

Writing experiences at Saint Mary's College 



To what extent did you instructors provide specific guidelines for their writing 
assignments? 

Never 0.8% 

Sometimes 30.6% 

Usually 50.8% 

Always 17.8% 

To what extent did your instructors emphasize revision? 
Never 7.3% 
Sometimes 33.1% 
Usually 31.6% 
Always 28.0% 

To what extent did your instructors provide specific feedback on your writing 
before the end of the semester? 

Never 4.1% 

Sometimes 24.3% 

Usually 37.0% 

Always 34.6% 

About how many long (10 or more pages) papers did you write? 
Median 5 papers 

Mean 8.37 (Std Dev 7.41) 

Range 0 to 72 

About how many short (fewer than 10 pages) papers did you write? 
Median 1 5 papers 

Mean 23.71 (Std Dev 15.33) 

Range 0 to 100 



percent) of the instructors "usually" or "always" provided specific guidelines for 
their writing assignments, and nearly all the rest "sometimes" did so. Well over 
half (59.6 percent) of the instructors also emphasized revisions, and even more 
(71.6 percent) provided specific feedback on their writing during the course of the 
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Table 6 

Evaluation of Writing Experiences at Saint Mary's College 



How did your teachers provide writing instruction? (check all that apply) 



Comments written on paper 594 

Written guidelines 490 

Suggestions during revision 484 

Lecture in class 431 

Peer feedback 237 

Other 37 



In what areas could you have used more instruction? (check all that apply) 

Professional format 302 

Grammar/style 270 

Personal/expressive writing 204 

Technical reports 191 

Library work 1 21 

Narrative writing 81 

Other 26 

I learned the most about writing when the assignment was , . . 



Mean 

4.11 

4.42 

2.29 

3.19 

3.65 

5.50 



short 
assigned topic 
draft revision 
in major 
in minor 
in-class assgnmt 
with prior class 



. . ; ; long 

..;....! you selected topic 

. . I . . . . ; no drafts required 

I .... I not in major 

. . . i . . . . I not in minor 

.X. i . . . . 1 take-home assgnmt 
without prior class 



2.94 discussion of topic | 1 x ; i 1 i discussions of topic 



class. 



The respondents indicated they had to write several (generally between five 
and eight) long papers of ten or more pages, and even more often (15 to 25 times) 
were required to write a shorter paper. 
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Finally, the survey asked respondents to indicate their recollection of how 
writing instruction was provided in class, and the areas in which they now feel 
they could have used more instruction. These results are shown in Table 6. I 

The method utilized by teachers in providing writing instruction is listed in 
ranked order. The method utilized most often was comments written on the paper. 
Written guidelines were also widely used, as were suggestions during the revision 
process and class lectures. Utilized less frequently (but still for more than a third 
of the respondents) was peer feedback. 

Wider experience in using professional format, and the need for sharpening 
grammar and style skills were cited most often as the areas where the respondents 
felt they could have used more instruction. Speciolized writing, such as technical 
reports and personal/expressive writing, were also cited by nearly a third of the 
respondents. 

The characteristics of the learning situation where respondents learned the 
most about writing were take-home assignments that included draft revisions with 
prior class discussion of the topic. Of the seven paired dichotomies, the results for 
these three characteristics were the furthest from the center of the seven-point 
semantic-differential-type scale. Typically these experiences were in courses 
related to the students' majors. Slight preferences were stated for papers with 
writer-selected topics (as opposed to teacher-assigned topics), papers assigned in 
classes related to the students' minor (as opposed to not being in the minor), and 
longer papers. 

^ lb 
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As the faculty reviewed the results of this survey, they noted a couple of 
items where the responses were ambiguous and difficult to explain. The finding 
that many respondents wished they could have had more instruction in grammar/ 
style surprised the faculty, but they were at a loss from the responses that were 
made as to whether this need was for more grammar instruction or style 
instruction, or whether something else was being communicated by the 
respondents. The faculty would like to see further investigation to clarify this area. 

It was also noted in the analysis of the results grouped by the respondents' 
major that the science majors tended to report having written fewer papers in their 
studies at Saint Mary's, Apparently these respondents were not including in their 
tallies the many written laboratory reports they had completed. Future studies 
might indicate parenthetically the types of assignments typically included in the 
characterization of long and short papers. 

Discussion 

The fact that there is no comparative information for this survey makes it 
difficult to ascertain whether or not these results are typical for all college 
graduates, or whether the Writing Program indeed made a difference in the post- 
baccalaureate experience of Saint Mary's graduates. Yet, there is ample evidence 
that writing remains an important part of these graduates' lives, and there is a 
wealth of evidence related to the types of writing being done in the data that was 
not reported in this paper. This information, particularly, will be helpful to the 
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Writing Program faculty as they develop real-life assignments to better prepare 
current and future students for life after college. 

The evidence presented also gives an indication of the writing experiences 
that have been most effective in preparing the graduates for their present 
situations, and this can assist the faculty in further shaping these experiences to be 
most helpful. Analysis of these results by the graduation year of the respondent 
can also tell whether the later implementation of the "Advanced W" was perceived 
as being more helpful to those students who experienced this particular activity. 

The survey provides clear evidence that the Writing Program pervades the 
entire curriculum at Saint Mary's, as it appears only a minuscule proportion of 
faculty never utilize writing-related activities. The large number of assigned papers 
(an average of 30 short and long papers per respondent) indicates there are indeed 
few classes without writing assignments. 

The respondents also provided feedback regarding the areas in which they 
could have used more instruction, and the Writing Faculty will give attention to 
these recommendations. 

Conclusions 

Can a survey of a writing-across-the-curriculum program provide evidence 
usable in assessing the academic program of a college or university? The evidence 
presented here affirms the usefulness of such a survey, and gives some indication 
of the kinds of questions that can be asked in this type of survey. 
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Clearly, comparative data from similar and different types of schools would 
provide a better assessment, and it is hoped that as writing-across-the-curriculum 
program become more widespread, collaborative evaluations involving the sharing 
of these types of results will yield comparable information. 

The information gathered with this survey has proven quite valuable at Saint 
Mary's College in identifying the type of writing activities experienced by their 
graduates after leaving college, and also in pinpointing areas where the program 
can be strengthened. 

It is hoped the results presented here can contribute in a small way to the 
growing body of research on the effect of writing-across-the-curriculum programs. 
Institutional researchers from campuses incorporating this type of writing program, 
and those where this curricular innovation is being contemplated, can benefit from 
the information presented and the recommendations suggested in this paper. 
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